The SmartKom Architecture: A Framework for Multimodal Dialogue Systems
نویسندگان
چکیده
SmartKom provides an adaptive and reusable dialogue shell for multimodal interaction, which has been employed successfully to realize fully-fledged prototype systems for various application scenarios. Taking the perspective of system architects, we will give a review of the overall design and specific architecture framework being applied within SmartKom. The basic design principles underlying our approach will be described and the different instantiations of the conceptual architecture will be presented to illustrate the adaptiblity and flexibility of the generic framework.
منابع مشابه
Building Multimodal Dialogue Applications: System Integration in SmartKom
In this contribution, we will report on the experience gained in building large-scale research prototypes of fully integrated multimodal dialogue systems in the context of the SmartKom project. The development of such systems requires a flexible software architecture and adequate software support to cope with the challenge of system integration. A practical result of our experimental work is an...
متن کاملDialogue Systems Go Multimodal: The SmartKom Experience
Multimodal dialogue systems exploit one of the major characteristics of humanhuman interaction: the coordinated use of different modalities. Allowing all of the modalities to refer to and depend upon each other is a key to the richness of multimodal communication. We introduce the notion of symmetric multimodality for dialogue systems in which all input modes (e.g., speech, gesture, facial expr...
متن کاملSmartKom: Symmetric Multimodality in an Adaptive and Reusable Dialogue Shell
We introduce the notion of symmetric multimodality for dialogue systems in which all input modes (eg. speech, gesture, facial expression) are also available for output, and vice versa. A dialogue system with symmetric multimodality must not only understand and represent the user's multimodal input, but also its own multimodal output. We present the SmartKom system, that provides full symmetric ...
متن کاملMobile Multimodal Dialogue Systems
Mobile multimodal dialogue systems allow the user and the system to adapt their choice of input and output modality according to various technical and cognitive resource limitations and the task at hand. We present the multimodal dialogue system SmartKom, that can be used as mobile travel companion for car drivers and pedestrians. SmartKom combines speech, gestures, and facial expressions for i...
متن کاملSmartKom: Towards Multimodal Dialogues with Anthropomorphic Interface Agents
SmartKom is a multimodal dialogue system that combines speech, gesture, and facial expressions for input and output. SmartKom provides an anthropomorphic and affective user interface through its personification of an interface agent. Understanding of spontaneous speech is combined with video-based recognition of natural gestures and facial expressions. One of the major scientific goals of Smart...
متن کامل